The Analytic Hierarchy Process and Data-less Prediction

Building useful effort prediction systems for software engineering projects is difficult in the absence of historical data. To overcome this problem we propose the use of Saaty’s Analytic Hierarchy Process (AHP) to elicit subjective expert opinion using pairwise comparisons. We briefly describe the theory of AHP and then outline how it might be harnessed to overcome a lack of historical data. Two case studies, using data from BT and another telecommunications company, illustrate the method’s potential. If all the comparisons are correctly made, extremely accurate results are possible (of the order of mean magnitude of relative errors (MMRE) =5%). The approach is also very tolerant to incorrect comparisons: even when 40% of all comparisons are randomised, the accuracy is still much better than a linear regression model or expert judgement. Next we describe a simple pilot study which indicates that project managers are capable of making the pairwise judgements our method requires. We then consider some problems that need to be overcome before this approach can easily be deployed. These include lack of homogeneity in the elements being compared and excessive numbers of comparisons. We believe this novel effort prediction approach has given encouraging results and warrants further investigation.